Each year, traffic over computer networks such as the Internet gets heavier. Typically, rather than having a single server handle all requests for similar information, multiple servers are used. For example, a web site may have many servers that are capable of servicing incoming requests. At essentially one point in time, there may be hundreds of users making requests to the web site. A front-end server distributes those requests among the servers as best as it can. However, conventionally, the distribution of requests is not always efficient.
The front-end server, which may be referred to as a load balancer, can use one of several common techniques to balance the load among the back-end servers. The simplest technique is round robin. A round robin system simply rotates the requests to the back-end servers with each back-end server getting one request per rotation. Another common load balancing technique is to assign the request to the back-end servers randomly. Over the long haul, each back-end server should receive an equal number of requests. Still another technique is to assign a number to each of “N” servers and forward the request to a server by determining a number between 1 and N for each request. The request is then forwarded to the back-end server whose number matches the request number. Unfortunately, each of these techniques suffers in that imbalances in load can occur for a variety of reasons.
A second problem with conventional load balancing is that one of the back-end servers can go down. The front-end server sometimes does not recognize this, which results in a request being forwarded to a server that is unable to handle the request. This typically results in an error message and a frustrated requestor. For example, the request may witness the all too common error message that information is not available. If the user tries to get the information again the problem may go away simply because by chance the request is forwarded to a functioning server this time. However, the user has wasted time and raised his/her frustration level. It is also possible that the error is only logged locally, and the user's browser sits waiting for a response that never arrives.
Thus, one problem with conventional load balancing is that the load becomes imbalanced between the servers. Another problem with some conventional load balancing techniques is that they fail to recognize when a server has gone down.